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1 . Applicant's indicates that the election of invention group I that was made prior to 
previous office action was made with traverse, with no additional remarks presented. 
However, the Examiner deems that the two additional inventions that were not elected 
would represent an undue burden to examine with the one elected, owing to the 
separate utility found in the subcombination claims. Therefore, the restriction 
requirement remains, and is made FINAL. 

2. The text of those sections of Title 35, U.S. Code not included in this action can 
be found in a prior Office action. 

3. Claims 45 - 47 are rejected under 35 U.S.C. 101 because the claimed invention 
is directed to non-statutory subject matter. 

In an attempt to alleviate the difficulty caused by claims directed, simply, to a 
"data packet" and "memory" (these not referring to one of the statutory classes of 
invention but instead to information perse), applicant has introduced the phrase 
"computer-readable medium having computer executable instructions for employing a" 
in each of the claims. 

However, this does not result in a statutory "machine" or "article of manufacture" 
claim, when the claim is considered in light of applicant's disclosure. At page 6, line 24, 
applicant specifies that "computer readable media" can include "carrier waves". These 
"carrier waves", rather than tangibly embodying the information, are merely a transient 
phenomenon in which the information signal is non-tangibly fixed. Thus, the claims 
remain directed to information per se. 
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4. Claims 1 , 4 - 6, 8 - 10, 43, 44, 52, 53 are rejected under 35 U.S.C. 102(e) as 
being anticipated by Jain et al ("Jain"; U.S. Patent Number 6,567,980). 

As per independent claim 1 (and see also independent claim 6), Jain discloses a 
VIDEO CATALOGER SYSTEM that facilitates non-linear viewing of media, the system 
comprising: 

a scene selector that scans a digitized media and selects a scene in the digitized 
media (col. 2 lines 10-19): Using advanced media analysis algorithms 
that automatically watch, listen to and read a video stream, the 
multimedia cataloger intelligently extracts metadata-keyframes , 
time codes, textual information and an audio profile from the 
video in real-time . The media analysis in Jain uses "at least one of... item 
recognition", to satisfy the alternatives list in the claim. The algorithms serve to "select" 
digitized media for various categories of inclusion. 

Jain then teaches a metadata generator that produces metadata associated with 
the selected scene and relates the metadata to the selected scene (col. 2 lines 6-9): 
[T]here is a media cataloging and media analysis application 
which performs real-time, or non- real -time, indexing and 
distribution of video, across an enterprise. 

The result is that Jain provides an organizer that places the selected scene and 
the metadata in a media store to facilitate non-linear viewing of one or more scenes 
(col. 2 lines 20-23) : This information becomes the foundation of a 
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rich, frame-accurate index that provides immediate, non-linear 
access to any segment of the video. 

As per claim 4, Jain discloses that the recognition is adapted by a machine 
learning technique based, at least in part, on the input from the user (Table 1 col. 7; col. 
7 lines 7-12): unique metadata can be defined and added to the Video 
Cataloger 110 by a user . Custom metadata tracks could include 
information provided in collateral data to the video. These 
metadata tags allow the user to search digital videos, and they are inputted by the user, 
after which the computer has "learned" them. 

Concerning claim 5, Jain discloses the metadata generator produces at least 
one of... an item name (col. 6 lines 2-5): The vidsync daemons also are 
responsible for returning certain pieces of information to the 
Cataloger, such as the actual start time, and a digital video 
asset ID or name for later use. 

As per claim 8, Jain discloses selecting the scene from the set of digitized media 
by the above-noted analysis that is applied to the video that is received for cataloging. 
This results in automatically selecting the scene based on at least one of... item 
recognition and, as in claim 9, in associating at least one of... an item... with the selected 
scene . 

Concerning claim 10 (and see also claim 36), Jain teaches at least one of, a 
database and a datacube (col. 3 lines 60-67): Metadata Server 130: may be as 
simple as a file system containing hypertext markup language 
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(HTML) files, or as complex as a relational database supporting 
a client -server application environment for media management. 

As per independent claim 43, Jain necessarily involves data structure that 
facilitates non-linear viewing of media items, the data structure comprising: 
a first field that holds a media item (to generate Figure 2's item 176); and 
a second field that holds a metadata item related to the media item, where the 
metadata facilitates locating a related media item (as in Figure 2 item 178), by at least 
one of identifying the media item. The metadata serves both the purposes of 
"identifying" (claim 52) and "locating the media item" (claim 53). 

Concerning independent claim 44, Jain also involves a method of providing and 
selecting from a set of graphical user interface elements on the display in returning 
desired content from the cataloged collection, the method comprising: 

retrieving a set of graphical user interface elements, each of the interface 
elements representing an action associated with facilitating the non-linear display of 
media items (col. 2 lines 20-24): This information becomes the foundation 
of a rich, frame-accurate index that provides immediate, non- 
linear access to any segment of the video.; 

displaying the set of interface elements on the display (Figure 2); 
receiving an interface element selection signal indicative selecting a selected 
entry from the set of interface elements (col. 4 lines 26-28): A panel 172 displays 
the live video being digitized, with play, stop, etc. controls 
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that interact remotely with the analog source via a deck 

controller 240 (FIG. 3) . ; and 

initiating processing to facilitate non-linear viewing of media based, at least in 
part, upon stored metadata, this being the associated metadata that is used in Jain's 
catalog . 

5. Claims 45-47 are rejected under 35 U.S.C. 102(e) as being anticipated by Morris 
("Morris"; U.S. Patent Application: 2002/0088000). 

As per independent claim 45 (and see also independent claim 46), Morris, in 
providing CONTROLED ACCESS TO IMAGE METADATA , uses a data packet adapted 
to be transmitted between two or more computer components that facilitate annotating a 
media and facilitate the non-linear viewing of the media, the data packet comprising: 
a first field that stores a clip identifier (Figure 2 item 52), in that the header of a 
data packet is used to identify that this is a new packet, and the packet contains new 
information of the media; 

a second field that stores a metadata key that identifies an annotating metadata 
associated with the clip identified by the clip identifier (Figure 2 items 60 & 62; 
paragraph 20 ): The metadata for the images is preferably stored 
within the image file 50 in individual image tags 60, each of 
which store various types of data that correspond and relate to 
particular captured image data 54.; and 

a third field that stores data associated with the clip identified by the clip identifier 
(Figure 2 item 54)— this is the field where the actual data is stored. 
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As per independent claim 47 in addition to the above-noted "first field" for a "clip 
identifier" and "field that stores metadata", Morris's data arrangement also uses a field 
that stores a requested user action concerning the portion identified by the clip identifier 
(Figure 2 items 60 & 62; paragraph 20), since in transmitting a request for a metadata- 
annotated item in Morris, "user action" will also be specified. 
6. Claims 7,15- 20, 30 - 40 are rejected under 35 U.S.C. 1 03(a) as being 
unpatentable over Jain in view of Duncombe et al ("Duncombe"; U.S. Patent Number 
6,813,745). 

As per claim 7, Jain discloses a method for cataloging digital media into a 
storage device to allow the user to retrieve non-linear clips. This allows the user to 
quickly search and find digital data. 

The difference between the claims and Jain is the claims recite "manually 
scanning one or more scenes from the set of digitized media; and manually selecting 
the scene." 

Duncombe teaches a media system for storing electrical files that allow the user 
to enter search descriptions for digital retrieval similar to that of Jain. In addition, 
Duncombe discloses (col. 3 lines 13-14): FIG. 8 is a flow diagram showing 
the steps taken by a user to select and view a plurality of 
suitable media clips. 

It would have been obvious to one of ordinary skill in the art to modify the viewing 
GUI taught by Jain to permit the user to perform manual selections of the media file as 
per Duncombe, in order to obtain a system that automatically or manually allows the 
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user to manipulate and operate a digital media storage, thus improving flexibility and 
user options available. 

Concerning independent claim 15, Jain discloses a method for cataloging digital 
media into a storage device to allow the user to retrieve clips in a non-linear fashion. 
This allows the user to quickly search and find digital data. In addition, Jain teaches a 
scene retriever that retrieves one or more scenes and one or more pieces of 
annotating metadata associated with the one or more scenes from a media store, as 
per the metadata annotations, provided by a metadata analyzer that can identify one or 
more relationships between the one or more pieces of annotating metadata (col. 2 lines 
6-9 & col. 2 lines 27-30): In one aspect of the invention, there is a 
media cataloging and media analysis application which performs 
real-time, or non- real -time, indexing and distribution of video 
across an enterprise. Synchronized encoding and indexing allows 
users to intelligently navigate through the video by using the 
index to go directly to the exact point of interest, rather than 
streaming it from start to finish. The metadata serves to find media in 
Jain that are of a similar nature, and thus, have relationships between them. 

The difference between the claims and Jain is the claims recite "a playlist 
generator that evaluates the one or more relationships and produces a playlist of related 
scenes, one or more viewers for viewing a scene listed in the playlist, one or more 
feedback receivers for receiving a feedback concerning the viewed scene; and a playlist 
updater for updating the playlist based, at least in part on the feedback." 
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Duncombe teaches a system that allows the user to browse through video clips 
by receiving user feedback, which is similar to that of Jain. In addition, Duncombe 
teaches a playlist generator that evaluates the one or more relationships and produces 
a playlist of related scenes (col. 2 lines 34-36): The method includes the steps 
of first organizing and formatting a plurality of media clips, one 
or more viewers for viewing a scene listed in the playlist (col. 2 lines 35-36): and then 
playing the plurality of media clips based upon input supplied 
by the user . , one or more feedback receivers for receiving a feedback concerning 
the viewed scene to direct a playlist updater (col. 2 lines 50-55): Once the 
plurality of suitable media clips have been played, the media 
system uses a means for soliciting feedback and a means for 
accepting feedback from the user, and a means for reselecting 
the plurality of suitable media clips based upon the feedback of 
the user. 

It would have been obvious to one of ordinary skill in the art at the time the 
invention was made, to modify the viewing GUI taught by Jain to allow the user to 
search media files with the playlist and feedback control of Duncombe, in order to obtain 
a system that has the ability to generate a set of digital media clips that are in better 
accord with actual user desires. 

Concerning claims 16, 38, Jain/Duncombe discloses output on at least one of, an 
active device and a passive device (Duncombe, Figure 1 item 90, Figure 2 item 90, 
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Jain, Figure 2). As per claim 17, the viewing arrangements of Jain/Duncombe are also 
at least one of, an intelligent device and a non-intelligent device (see also claim 37). 

The input in claim 18 that is at least one of, a touch input, a typed input, a mouse 
input, a voice input and a facial expression input concerning the viewed scene reads 
upon Duncombe (col. 6 lines 53-60): a data input mechanism 72, such as a 
keyboard 72 and a mouse 72A. A similar line of reasoning applies to claim 40. 

As per claim 19, Jain in view of Duncombe discloses that the feedback concerns 
a current scene and also that the feedback comprises at least one of, a command to 
skip ahead in the playlist, a command to skip back in the playlist, a command to 
generate a new playlist, a command to find scenes similar to the current scene and a 
command to play a longer scene related to the current scene (Jain, col. 4 lines 27-30): 
A panel 172 displays the live video being digitized, with play, 
stop, etc. controls that interact remotely with the analog 
source via a deck controller 240 (FIG. 3). Responsive to such 
feedback, the playlist updater of Duncombe adds and/or removes a scene from the 
playlist based on at least one of, a usage data, a feedback command and a time stamp 
as in claim 20 (Duncombe, Figure 2 item 1 12, or Figure 13). 

As per independent claim 30, the Jain "media database" as played by the 
Duncombe "playlist" would be populated by "video segments" being "associated with 
annotating metadata" under the control of an "annotating tool". Such Jain-style 
annotation will be "based on at least one of... item recognition", so that the annotating 
metadata comprises at least one of... an item identifier, as in claim 31. 
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As per claim 32, the production of annotation data in response to user input is 
at least provided by the user-operable catalog creation that is seen in Jain. As in claim 
33, the resulting metadata is, at least in part... an item recognition data (Jain, col. 6 lines 
56-60): Likewise, the remaining metadata tracks (Audio Classes 
324, Speech 326, Speaker ID 328, Keywords 330) are each a parcel 
of metadata spanning a time period, and are extracted by their 
corresponding feature extractor shown in FIG. 9. The input thus 
supplied is made part of "a machine learning technique", where the machine learns of 
user's intentions as to the media information (claim 34). 

As per independent claim 35, Jain teaches a media data store comprising one or 
more metadata-annotated, displayable items (Figure 2 item 178), and one that operates 
in conjunction with a presenter that presents a selected first displayable item from the 
media data store (col. 4 lines 25-30). 

The difference between the claim and Jain rests in the "selector that selects a 
second displayable item from the media data store based, at least in part, on a 
relationship between a first metadata associated with the first displayed item and a 
second metadata associated with the second displayable item." 

Duncombe teaches a system that allows the user to browser through video clips 
and that selects a second displayable item from the media data store based, at least in 
part, on a relationship between a first metadata associated with the first displayed item 
and a second metadata associated with the second displayable item (col. 2 lines 65-67): 
A further objective is to provide a media system that solicits 
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feedback from the user and modifies the media presentation in 
response to the feedback. The specifics of user desire in Duncombe create a 
relationship between what is seen first and what is seen second. 

It would have been obvious to one of ordinary skill in the art at the time the 
invention was made to modify the viewing GUI taught by Jain to include a relationship- 
establishing playback arrangement as per Duncombe, in order to obtain a system that 
will retrieve digital media segments in better accordance with user preference. This will 
then handle the situation of retrieving the second displayable item in response to the 
user's feedback concerning the current item (claim 39). 

7. Applicant's arguments filed 21 March 2005 have been fully considered but they 
are not persuasive. 

At page 16 of the remarks applicant argues, concerning the rejection under 35 
USC 101, that "the claimed invention reduces to a practical application that produces a 
useful, concrete, tangible result", which is in accordance with AT&T v. Excel as cited. 
However, the test in this situation is not whether the claimed invention has practical use 
or tangible ultimate results when implemented, but whether the "computer-readable 
medium" is in fact properly found within the four statutory classes of invention. Since 
the specification permits a "carrier wave", this is not tangible fixation in the kind of 
structure that would pass muster under the "process" or "manufacture" categories. 

As per the applicability of Jain, applicant argues (pages 17-18) that "what is 
disclosed is merely a means of cataloging all media, not selecting scenes based upon 
criteria of the digital media". However, the Jain use of media analysis indeed is one in 
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which a selection process occurs, and one that is directed towards making the media 
collection properly searchable by an index. The Examiner also notes that the claims do 
not actually recite that the invention "can continuously analyze the media to facilitate 
intelligent search and retrieval". The Examiner cannot read in the amount applicant 
would seek to a phrase such as "selects a scene". 

As per independent claims 43, 44, applicant argues at page 19 that "once the 
metadata is generated" in Jain, "subsequent searches for related media only entail a 
search of the metadata, not the media itself. However, all that actually appears is that 
"the metadata facilitates locating a related media item". This is met by Jain, who 
annotate the media with metadata for the purpose of initiating a search that locates the 
media information on the basis of such metadata, and in accordance with the user's 
indications (as particularly appears in independent claim 44). 

As per Morris, applicant argues (page 20) that "Morris only stores metadata, not 
a metadata key that identifies an annotating metadata associated with the clip 
identified with the clip identifier". However, the metadata storage in Morris is sufficient 
to read upon the "metadata key" because it is simply an identification of metadata, and 
is also a "key" for further user selection, in those transmissions of commands that would 
involve "a requested user action". 

As per the combination of Jain and Duncombe, applicant argues (pages 21 -22) 
that "Duncombe only compares media with predetermined values; it does not evaluate 
relationships between the metadata". However, in permitting user-directed, criteria- 
responsive searches, Duncombe will in fact consider a "relationship" between data 
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items, such as those that can form the sequential "playlist of related scenes" that 
applicant alleges is missing at page 22. The assignment of metadata as per Jain will 
work in direct cooperation with such a search, and produce a series of "first" and 
"second" items whose access is directly formed by user desires. 

8. THIS ACTION IS MADE FINAL Applicant is reminded of the extension of time 
policy as set forth in 37 CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1 .136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the mailing date of this final action. 

9. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Raymond J. Bayerl whose telephone number is (571) 
272-4045. The examiner can normally be reached on M - Th from 9:00 AM to 4:00 PM 
ET. 

10. If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, John Cabeca, can be reached on (571) 272-4048. All patent application 
related correspondence transmitted by FAX must be directed to the central FAX 
number (703) 872-9306. 
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1 1 . Any inquiry of a general nature or relating to the status of this application or 
proceeding should be directed to the receptionist whose telephone number is (571) 272- 
2100. 
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